Experiments in Single and Multi-Document Summarization Using MEAD

In this paper, we describe four experiments in text summarization. The first experiment involves the automatic creation of 120 multi-document summaries and 308 single-document summaries from a set of 30 clusters of related documents. We present official results from a multi-site manual evaluation of the quality of the summaries. The second experiment is about the identification by human subjects of cross-document structural relationships such as identity, paraphrase, elaboration, and fulfillment. The third experiment focuses on a particular cross-document structural relationship, namely subsumption. The last experiment asks human judges to determine which of the input articles in a given cluster were used to produce individual sentences of a manual summary. We present numerical evaluations of all four experiments. All automatic summaries have been produced by MEAD, a flexible summarization system under development at the University of Michigan.